Reproducing Statistical Results

The reproducibility of statistical findings has become a concern not only for statisticians, but for all researchers engaged in empirical discovery. Section 2 of this article identifies key reasons statistical findings may not replicate, including power and sampling issues; misapplication of statistical tests; the instability of findings under reasonable perturbations of data or models; lack of access to methods, data, or equipment; and cultural barriers such as researcher incentives and rewards. Section 3discusses five proposed remedies for these replication failures: improved prepublication and postpublication validation of findings; the complete disclosure of research steps; assessment of the stability of statistical findings; providing access to digital research objects, in particular data and software; and ensuring these objects are legally reusable. 1 A nn ua l R ev ie w o f St at is tic s an d It s A pp lic at io n 20 15 .2 :1 -1 9. D ow nl oa de d fr om w w w .a nn ua lr ev ie w s. or g A cc es s pr ov id ed b y U ni ve rs ity o f Il lin oi s U rb an a C ha m pa ig n on 0 6/ 30 /1 5. F or p er so na l u se o nl y. ST02CH01-Stodden ARI 27 March 2015 12:18